Achieving focus in a digital pathology system

ABSTRACT

Methods and apparatus are provided for computing focus information prior to scanning digital microscope slide data with a line scan camera. The methods include a point-focus procedure that works by moving the slide to the desired measurement location, moving the objective lens through a predefined set of height values, acquiring imagery data at each height, and determining the height of maximum contrast. The methods also include a ribbon-focus procedure whereby imagery data are acquired continuously, while the slide and objective lens are in motion. Both methods may be applied with either a static or a dynamic implementation.

RELATED APPLICATION

The present application is a continuation of U.S. patent applicationSer. No. 12/423,571 filed 14 Apr. 2009, which is a continuation of U.S.patent application Ser. No. 10/827,207 filed 16 Apr. 2004 that claimspriority to U.S. provisional patent application Ser. No. 60/463,909filed on 17 Apr. 2003, where application Ser. No. 10/827,207 is acontinuation-in-part of U.S. Pat. No. 6,917,696 filed on Mar. 11, 2004,which is a continuation of U.S. Pat. No. 6,711,283 filed on May 3, 2000,each of which is incorporated herein by reference in its entirety.

BACKGROUND

1. Field of the Invention

The present invention generally relates to the field of digitalmicroscopy and more particularly relates to the focusing of a line scancamera prior to and during the capture of imagery data from a specimenon a microscope slide.

2. Related Art

In conventional virtual microscopy systems, image tiling techniquesproduce individual image tiles that can be significantly out of focusover much of the image. An image tiling system is restricted to a singlefocal distance for each individual snapshot taken by its camera, thus,each of these “fields of view” have areas that are out of focus when thesubject specimen being scanned does not have a uniform surface. At thehigh magnification levels employed in virtual microscopy, specimens witha uniform surface are extremely rare.

Conventional image tiling solutions are severely handicapped by theselimitations, with their only recourse being to discard a significantamount of out of focus image data, resulting in an increased number ofimage tiles that must be scanned and a corresponding increase in thetime to scan a microscope slide. Even so, the resulting image data stillsuffers from out of focus areas on each image tile. The discarding ofperimeter image data that is extremely out of focus still leaves out offocus image data in the image tile resulting from the inherent circularoptical distortion.

Recently, new line scan camera systems have been introduced to thevirtual microscopy industry such as the ScanScope® scanner created byAperio Technologies, Inc. The revolutionary ScanScope® scanner systemdoes not suffer from circular optical distortion due to its use of aline scan camera. Additionally, the line scan camera can adjust itsfocus for each line of pixels that are captured when scanning amicroscope slide. Thus, the quality of the resulting image from a linescan camera system is significantly better due to the sharp focus ofeach line of pixels captured by the line scan camera.

Accordingly, these significant advancements in the virtual microscopyindustry have created a need for a system and method that overcomes thesignificant focusing problems inherent in conventional image tilingsystems and capitalizes on the focusing capabilities of therevolutionary line scan camera systems.

SUMMARY

Systems and methods are provided for computing focus information priorto scanning microscope slides with a line scan camera based digitalmicroscopy system. In a point-focus procedure, the line scan camerasystem first positions the slide at a desired measurement location,moves the objective lens through a predefined set of height values andacquires imagery data at each height, and then determines the height(Z-axis setting) of maximum contrast. The maximum contrast height isthen established as the optimal focus height. In a ribbon-focusprocedure, the line scan camera system acquires imagery datacontinuously while the slide and objective lens are in motion. The slidemoves through the scanning path and the objective lens changes focusheight in a sinusoidal fashion. The captured imagery data are analyzedand heights of maximum contrast are determined, which establishes theoptimal focus height along the scan path. Both methods may be applied ineither a static or a dynamic implementation.

BRIEF DESCRIPTION OF THE DRAWINGS

The details of the present invention, both as to its structure andoperation, may be gleaned in part by study of the accompanying drawings,in which like reference numerals refer to like parts, and in which:

FIG. 1 is a side view illustrating an example microscope slide andobjective lens during the slide scanning process according to anembodiment of the present invention;

FIG. 2 is a top-view schematic illustrating an example microscope slidehaving a tissue sample, the area to be scanned, and a plurality of focuspoint locations according to an embodiment of the present invention;

FIGS. 3A-3B are graph diagrams illustrating course and fine incrementsfor determining the height profile for a stripe to be scanned accordingto an embodiment of the present invention;

FIG. 4 is a side view illustrating an example microscope slide andobjective lens during the ribbon-focus process according to anembodiment of the present invention;

FIG. 5 is a multi-graph diagram illustrating example imagery data andfocus point calculations for a microscope slide according to anembodiment of the present invention;

FIGS. 6A-6C are graph diagrams illustrating example focus pointmeasurements from the ribbon-focus procedure and the point-focusprocedure according to an embodiment of the present invention;

FIG. 7 is a block diagram illustrating an example scan area divided intoa plurality triangles according to an embodiment of the presentinvention;

FIG. 8 is a graph diagram illustrating an example focal surfaceaccording to an embodiment of the present invention;

FIG. 9 is a flow diagram illustrating an example process for pre-focusaccording to an embodiment of the present invention; and

FIG. 10 is a block diagram illustrating an exemplary computer system asmay be used in connection with various embodiments described herein.

DETAILED DESCRIPTION

Certain embodiments as disclosed herein provide a method for defining afocal surface corresponding to a specimen on a microscope slide prior toscanning the specimen with the line-scan camera of a virtual microscopysystem. For example, one method disclosed herein allows for theline-scan camera to scan imagery data at a plurality of focus points onthe specimen, with each focus point being scanned at a plurality ofheights of the objective lens. The resulting imagery data are analyzedto determine which frame of imagery data has the greatest contrast. Theheight of the objective lens having maximum contrast for that focuspoint location is then established as the focal height for that locationon the microscope slide. A focal surface is then computed from aplurality of focal heights. In subsequent scanning of the specimen onthe microscope slide, the line scan camera adjusts the height of theobjective lens in accordance with the focal surface resulting in avirtual slide image with optimal focus.

After reading this description it will become apparent to one skilled inthe art how to implement the invention in various alternativeembodiments and alternative applications. However, although variousembodiments of the present invention will be described herein, it isunderstood that these embodiments are presented by way of example only,and not limitation. As such, this detailed description of variousalternative embodiments should not be construed to limit the scope orbreadth of the present invention as set forth in the appended claims.

The ScanScope® scanner utilizes a line-scan camera, combined with amotion control system which moves a stage holding the microscope slideat constant velocity. The slide is moved in a direction that isorthogonal to an objective lens that is coupled with a line scan camera.The line scan camera is thus able to acquire fixed-width stripes ofimagery data as the slide moves beneath the lens. Very large areas maybe scanned by acquiring a number of overlapping stripes, which aresubsequently combined into a single composite image (also referred to asa virtual slide). When compared to the conventional image tilingapproach, the ScanScope® scanner is both faster and yields superiorimaging results, especially with respect to improved focus quality. Withproper calibration, it is possible to achieve near-perfect focus alongthe center of each stripe. The stripe widths are then adjusted tomaintain acceptable focus over the entire width, resulting in acomposite image that is uniformly well-focused.

Advantageously, certain pre-focus calibration procedures can beperformed prior to scanning in order to improve the scanning speed andthe overall focus of the resulting virtual slide image. It should benoted that the pre-focus calibration is not the same as certainauto-focus methods, in which the focus height for the objective lens isdetermined concurrent with the scanning process.

The pre-focus calibration procedure involves determining the properheight of the objective lens for bringing points on the slide intoproper focus (see FIG. 1). The sharpest focus will be obtained along theoptical axis, which is aligned with the midpoint of the line scancamera. In general, each point on the slide may require a differentfocal height to produce the sharpest image possible. Factors affectingfocal height include, but are not limited to: tilt and variability ofthe mechanical motion assembly (e.g., the stage), variations in thethickness of the glass slide and cover slip thickness, variations inspecimen (e.g., tissue) thickness, and thermal expansion. All but thefirst of these variations are different for each slide and require thatpre-focus calibration be performed as quickly as possible prior toscanning, so that the calibration data remain accurate during imageacquisition.

During the pre-focus calibration, the best-focus lens-height is measuredat a fixed number of points (“focus points”) on the slide. These pointmeasurements are then used to compute a focal surface, which is used toestimate the height of best-focus everywhere on the slide. During thescanning of a stripe, the height of the objective lens is constrained tofollow this pre-determined focal surface (see FIG. 1). As the accuracyof the focal surface improves with the number of pre-focus sample pointsmeasured, so does the quality of focus in the scanned imagery data.Provided enough sample points have been taken, each stripe will havenear-perfect focus along its midpoint. The width of the stripe isreduced, if necessary, in order to retain only the central portion ofthe stripe, having best focus quality.

FIG. 1 is a side view illustrating an example microscope slide andobjective lens during the slide scanning process according to anembodiment of the present invention. In the illustrated embodiment, theobjective lens preferably follows a path that is parallel to the tissuesurface in order to maintain a fixed focal distance. During a scan, theobjective lens moves up and down to follow the height of best focus.This height profile can be different for each stripe scanned.

There are alternative, yet complementary procedures for fast andaccurate focus point measurements. Point-focus and ribbon-focus are twosuch procedures. Both procedures are based upon the principal thatcontrast (differences in neighboring pixel intensities) is largest whenan image is in focus. Accordingly, imagery data are acquired atdifferent objective-lens heights and analyzed to determine the heightwhere there is maximum contrast. The two procedures differ in how thevertical motion of the objective lens is synchronized with thehorizontal motion of the slide during image acquisition.

The point-focus procedure works by moving the slide to the desiredmeasurement location, moving the objective lens through a predefined setof height values, acquiring imagery data at each height, and determiningthe height of maximum contrast. This stop-and-go process (motionfollowed by measurement) is repeated for a sequence of predefined focuspoints. Since motion and image acquisition occur sequentially, the timerequired for the point-focus procedure is the sum of motion and imageacquisition times. In practice, the total time is increased by the lagtimes associated with starting/stopping both the motion control systemand the camera frame-capture. These lag times occur for each heightvalue at every focus point.

In the ribbon-focus procedure, imagery data are acquired continuously,while the slide and objective lens are in motion. There is no waitingfor the slide/objective to get to a particular location/height beforeacquiring imagery data; instead, times of maximum contrast in theimagery data are related to position and height using the prescribedmotion profile. Since the ribbon-focus procedure executes motion andimage acquisition in parallel the time required will be determined bythe slower of these two processes, not their sum. The lag timesencountered in the point-focus procedure are also eliminated. Thesefactors make ribbon-focus inherently faster than point-focus for makinga large number of focus point measurements.

There are two implementations of these procedures: (1) static; and (2)dynamic. In the static implementation, all focus points for the entireslide (or entire specimen area on the slide) are measured before actualscanning begins. In the dynamic implementation, the focus pointmeasurements are interleaved between stripe scanning. The dynamicimplementation minimizes potential drift errors, since focus values aremeasured as they are needed during the stripe scanning sequence. Thedynamic implementation may also make more efficient use of computerresources, since the focusing tasks can be performed in parallel withother scanning tasks.

There are many benefits of the pre-focus calibration procedures. Abenefit of the point-focus procedure is that it can be used to focus aselect number of optimally placed points. The focusing of each pointproceeds independently, without any information about the focus heightelsewhere on the slide. This enables focusing to be done where it isneeded (e.g., on the specimen) and avoids wasting time on the unusedclear area of the slide. Advantageously certain software routines areavailable to locate the tissue-containing regions on the slide anddetermine the focus point locations for the point-focus procedure.

In one embodiment, the point-focus procedure can be optimized forsampling the focus height. For example, a coarse height-sampling acrossa wide vertical range is first used to narrow the range where image datais to be captured and analyzed. Finer increments within the range arethen used, with successive narrowing of the range in order to achievehigh precision with the fewest number of image acquisitions at differentheights for the particular focus point. Current hardware requiresapproximately 1.0 second per focus point using the optimized point-focusprocedure.

The time for the ribbon-focus procedure is approximately 0.1 seconds perfocus point. This time is determined by the existing vertical motioncontrol, which is limited to 10 Hz peak-to-peak cycling—faster focusingtimes are possible with improvements to the vertical positioning system.At this rate, a 15 mm by 15 mm scan area can be sampled at 0.5 mmincrements, for a total of 30×30=900 sample points, in 90 seconds (1.5minutes). This time compares favorably with the scan time of 5 minutesfor an area of this size.

The methodology used to generate the focal surface from the list ofmeasured focus points is quite general and able to handle a large numberof points at irregularly spaced locations. This is necessary, due to thefact that the locations of specimen material on the slide can be quiteirregular. Thus, a uniform distribution of focus points on the slide isunlikely for tissue slides. The ribbon-focus procedure also generates anon-uniform distribution of sample points.

The dynamic implementation reduces the time between focus pointmeasurement and the actual scanning of stripes which depend upon thosefocus values. Focus points are identified and measured as they areneeded for the next stripe to be scanned, minimizing thermal drifteffects and allowing accurate focus to be maintained over the entireslide. In addition, the dynamic implementation may not add directly tothe overall scan time, since focusing may be done in parallel with otherscanning tasks (such as compression and disk input/output), whichutilize a different set of hardware resources.

Advantageously, both point-focus and ribbon-focus procedures will workwell on various slide types including tissue, cytology, and TMA. Forcytology and TMA slides, where large numbers of focus values must bemeasured in order to capture the variability in focal height,ribbon-focus is the faster and preferred method. When fewer points areneeded, such as in a small number of isolated tissue groups, thepoint-focus method may be faster. When the number of required focuspoints is not known in advance, the ribbon-focus procedure is preferred,since it can provide the largest number of focus points in the shortestamount of time.

FIG. 2 is a top-view schematic illustrating an example microscope slidehaving a tissue sample, the area to be scanned, and a plurality of focuspoint locations according to an embodiment of the present invention.Before executing the point-focus procedure, the scan area and thelocations of potential focus points are determined (see FIG. 2). In oneembodiment, the focus points can be identified with a tissue finderprocedure that determines, for example, a polygon surrounding theperimeter edge of one or more tissue samples and a plurality of focuspoints within the tissue sample(s). For tissue slides, the scan area isoften a small sub-region of the slide, while for cytology, the scan areamay comprise the entire visible slide area. The density and number offocus points can be determined by operational parameters of the tissuefinder procedure.

A simple (although not necessarily the fastest) procedure fordetermining the optimal focal height for a given focus point is asfollows:

-   -   1. Position the slide by moving it horizontally (x and y) until        the objective lens is centered above the point to be focused    -   2. Position the objective lens at an extreme limit of vertical        travel (e.g., the bottom)    -   3. For each height value        -   i. Position objective lens at current height value        -   ii. Capture a frame of imagery data        -   iii. Calculate contrast at current height        -   iv. Increment height value    -   4. Compare contrast values for all height values

Alternatively, the procedure could store the first calculated contrastvalue and then compare each new contrast value to the stored value andstore the new value if it indicated greater contrast. Alternativemethods may also be employed, as will be understood by one having skillin the art. Additionally, the frame of imagery data that is captured isadvantageously discarded after the contrast value has been calculated.

Typical height sampling is over a range of 100 μm, at increments of 0.20μm, for a total of 500 height samples per focus point. This approach isvery time consuming, since the height sampling is done in discretesteps, each of which requires a finite amount of time for motion to becompleted, e.g, 10 seconds per focus point, which is about 20milliseconds for each height value. Accordingly, decreasing the numberof height values at which image data are captured and the contrast valueis calculated can increase the efficiency of the overall point-focusprocedure.

FIGS. 3A-3B are graph diagrams illustrating course and fine incrementsfor determining the height profile for a stripe to be scanned accordingto an embodiment of the present invention. A dramatic improvement inexecution time can be achieved by an interval refinement procedure, inwhich coarse increments are used for the initial height sampling,followed by a sampling at finer increments surrounding the point wherethe highest contrast was found in the previous sampling. The spacing ofsample height intervals in the initial course measurement (e.g., insequence 1) is preferably small enough to ensure that the high-contrastregion is not missed altogether. A spacing of 5 μm or less is reasonablefor pathology slides using 20× objective lenses. At highermagnification, this spacing may need to be reduced, due to a reductionin the depth of focus.

In FIG. 3A, sequence 1 comprises 20 height samples, at 5 μm increments,over the full 100 μm range. In the illustrated embodiment, the maximumcontrast for sequence 1 was found at 60 μm. Accordingly, sequence 2comprises 20 height samples in the 50-70 μm range. The maximum contrastfor sequence 2 was found at 64 μm. Thus, as can be seen in FIG. 3B,sequence 3 comprises 20 height samples in the range 62-66 μm. This lastsequence of samples takes place with an interval of 0.20 μm betweensample heights. As shown, the maximum contrast for sequence 3 was foundat 64.4 μm. Advantageously, the optimal focal height for the particularfocus point in this example was identified with only 60 sample heights.A uniform sampling procedure would require 500 sample heights toidentify the 64.4 μm maximum contrast height. In one embodiment, theinterval approach can be ten times faster than a uniform sampling at0.20 μm increments over the entire 100 μm height interval.

FIG. 4 is a side view illustrating an example microscope slide andobjective lens during the ribbon-focus process according to anembodiment of the present invention. In the ribbon-focus procedure, astripe of imagery data is collected with the objective lens travelingalong an oscillatory height profile as illustrated in the figure. As thelens moves up and down through successive oscillations, it crosses theactual best-focus height a number of times. At each crossing, theimagery data will be in sharp focus and the corresponding contrast valuefor the frame of imagery data will by high. Thus, the planar locationand vertical height values are recorded as a focus point measurement.The hardware motion control must be synchronized with the camera frameread-out, so that the location (x,y) and height (z) of the objectivelens is known for each frame of imagery data. This synchronization isnecessary in order make the connection between high-contrast imageframes and focus point coordinates.

The ribbon-focus procedure advantageously yields a set of focus pointsalong the optimal focus path for the stripe. The sample points are notequally spaced, but they are quite regular in the sense that each timethe objective lens travels from one extreme of its trajectory to theother, a focus point measurement will result. A possible exception tothis is when the trajectory is over clear glass or the specimen contrastis too low to be measured. In such cases, there may be gaps in thesequence of focus points along the optimal focus path.

For the dynamic implementation of the focusing procedure, each stripe isscanned twice. The first scan uses the ribbon-focus procedure foracquiring the focus point values and defining the optimal path of theobjective lens, and the second scan follows the optimal path andactually scans the imagery data, resulting in a stripe of well-focusedimagery data.

FIG. 5 is a multi-graph diagram illustrating example imagery data andfocus point calculations for a microscope slide according to anembodiment of the present invention. In the illustrated embodiment, eachvertical line of imagery data (a) corresponds to a particular time, forwhich the horizontal position and objective lens height are known. Asthe lens moves up and down (c), the imagery data (a) alternates betweenblurry and focused. The contrast function (b) is calculated from theimagery data by summing the squared-differences in neighboring pixelvalues for each frame (column) of imagery data. Advantageously, thecontrast function peaks when the imagery data are in focus. A verticalline has been drawn in (b) to note locations of peaks in the contrastfunction. The same peak locations are shown in (c), where thecorresponding lens-height value is noted. Each time the imagery data arein focus, a peak in the contrast function occurs and a focus pointmeasurement is made. A threshold can be set so that only peaks thatexceed a certain magnitude will be used as focus points, therebydiscarding poor quality measurements.

Although a sinusoidal lens-height function is shown in FIG. 4 and FIG.5, many other functions are possible, including triangular and saw-toothfunctions. The particular function chosen will be determined by matchingit to the capabilities of the motion control system. With theribbon-focus procedure, the rate at which focus points are acquired islimited by the frequency capabilities of the vertical motion controlsub-system, the camera frame-rate, and the scanning velocity. Theslowest of these components will be the limiting factor in determiningthe focus point acquisition rate. In one embodiment, a piezo-controlused for height-positioning is limited to 10 Hz, which results in 0.10seconds per focus point.

FIGS. 6A-6C are graph diagrams illustrating example focus pointmeasurements from the ribbon-focus procedure and the point-focusprocedure according to an embodiment of the present invention. Thepoint-focus and ribbon-focus procedures are alternate methods of makingthe same measurements. With this in mind, each can be used to confirmthe accuracy of the other. The ribbon-focus measurements for the testdata shown in FIG. 5 were compared to point-focus measurements on thesame slide area. Approximately 100 focus points were measured with eachprocedure. A graph of the results from the ribbon-focus procedure isshown in FIG. 6A. A graph of the results from the point-focus procedureis shown in FIG. 6B. Both procedures yield nearly identical results, asillustrated in the FIG. 6C graph, which shows the two sets of results ina single graph format.

FIG. 7 is a block diagram illustrating an example scan area divided intoa plurality of triangles according to an embodiment of the presentinvention. When scanning a stripe of imagery data, the objective lensfollows a path along the height of best focus. Since the focus height isknown only at the focus point locations the focus height elsewhere onthe slide is estimated. In the simplest case, when the focus pointsactually lie on the scan path, the height can be estimated using linearinterpolation. For example, the height can be estimated using straightline ramps between the measured height values.

In most cases, a more general two dimensional interpolation method isneeded. In one embodiment, the scan area is divided into a set oftriangles by connecting each focus point with a pair of neighboringfocus points. For example, Delaunay triangulation may advantageously beused to capitalize on the way it generates triangles that have largerinterior angles than other triangulation methods.

To calculate the focus height at a given point on the slide, thetriangle enclosing the coordinates for that point is identified. Aplanar surface connecting the height values for the vertices of thattriangle is then constructed. The desired focus height is obtained byprojecting the given point onto that planar surface. The focal surfaceis then a set of planar triangular facets, joined at the height valuesof the measured focus points, as illustrated in FIG. 8, which is a graphdiagram illustrating an example focal surface according to an embodimentof the present invention.

An alternative to interpolation is fitting a functional form to thesampled height measurements. Function-fitting approaches have theundesirable effect that an anomalous height value at one point on theslide can affect the height estimate elsewhere. Additionally,function-fitting requires that the actual form of the height function beknown in advance. Inaccurate specification of this functional form willresult in height estimation errors. In an embodiment where themeasurements of the focus points are known with good precision,interpolation is preferred over function-fitting.

FIG. 9 is a flow diagram illustrating an example process for pre-focusaccording to an embodiment of the present invention. Initially, in step200, when the slide is first loaded into the line scan camera system, alow-resolution picture is taken of the slide for the purpose of locatingthe tissue-containing regions. A rectangular area enclosing all tissueregions is calculated, and defines the area to be scanned. Fornon-tissue slides (cytology), the scan region may include the entireslide. The scan area and focus point placement may also be donemanually, or be indicated by a bar-code on the slide label. By limitingthe scan area to that part of the slide that contains tissue, overallscan time is reduced. This is a significant benefit to systemperformance.

Next, in step 210, one or more points are placed on each region forpotential focus height sampling. Preferably, these focus pointscorrespond to locations that actually contain tissue, since thepre-focusing process makes use of spatial-contrast detail in the image.The density of focus points within a given tissue region is normallypre-set to a value that yields a practical number of points for theentire scan area. For non-tissue (cytology) slides, the focus points maybe distributed uniformly over the scan area.

Next, in step 220, the objective lens is positioned above a focus pointand then brought into approximate focus at the center of the area to bescanned. In one embodiment, an operator may be prompted to manuallyadjust the initial focus. Alternatively, positioning systems can beemployed that allow the macro focus adjustment to be done automaticallyunder computer control.

In step 230, an approximation to the focal surface is calculated bypassing a plane through three of the previously defined focus points.All possible three-point combinations are considered and the combinationthat yields the largest-area triangle is chosen for the planecalculation. This method of selection provides points that are widelyseparated and located at different azimuths from the center of the scanarea. The focus height at each of the three points is determined usingthe point method. An alternative approach is to least-squares fit aplane to the focus values for all focus points. An advantage of usingonly three points is that it can be done much quicker.

The plane approximation has two main purposes. First, it is used toachieve approximate focus when performing the pixel-gain calibration instep 240. Second, it is used to estimate the nominal focus height duringpre-focus. By knowing the approximate focal height, the search range canbe narrowed, resulting in faster pre-focusing.

The purpose of the pixel-gain calibration in step 240 is to correct fornon-uniformity in optical illumination, as well as variations in pixelsensitivity (fixed pattern noise). In one embodiment, a clear area ofthe slide (glass and cover slip only) is identified and scanned,resulting in a large number of samples for each camera pixel. Thesesamples are then averaged to yield a single number for each pixel,referred to as the pixel response. Hardware camera gains, integrationtime, and light-source intensity are all adjusted so that the maximumpixel-response does not exceed 255 counts, the maximum allowable valuedue to 8-bit A/D conversion. Note that there are red, green, and bluepixels, each having a unique pixel-response value.

The pixel gain is calculated by dividing the number 240 by the pixelresponse. Pixels near the center of the optical axis (center of the linearray) will have a larger pixel sensitivity, since the illumination isbrightest at this point. Pixel gain will therefore generally increaseaway from the center of the array, in order to compensate for thisillumination fall-off. Image data are corrected by multiplying eachrecorded pixel value by the corresponding pixel gain. This correctionresults in 8-bit image data which is uniformly white in areas containingonly glass. Spatial contrast detail in areas containing tissue or othercellular material are then due to true image variation, and the effectsof fixed-pattern noise is substantially reduced. This improves theperformance of focusing algorithms, which analyze spatial contrast indetermining focus height.

An alternative approach to measuring the pixel gain can be performed onnon-clear areas of the slide. This is particularly important forcytology slides, in which no clear area may be present. In thisapproach, the scan direction is parallel to the length of the line-scandetector array (normal scanning is perpendicular to the line array).Except for a small number of values near the beginning and end of thescan, this alternate scanning geometry results in each pixel recordingthe same sequence of image values. The ends can be trimmed appropriatelyusing cross-correlation techniques to define the extent of the trueoverlap-region for each pixel. The pixel response is calculated asbefore, by averaging the trimmed sequence of pixel values for each colorchannel. Rather than divide the pixel response into 240 as before, it isdivided into the average pixel response for the corresponding colorchannel. An additional calibration is needed to ensure that clear areasof the slide image are white in color.

With the static pre-focus procedure, all focusing is done prior toscanning of the slide. A type of focusing method is chosen as previouslydescribed, either ribbon-focus or point-focus. Using the chosen method,the focus height is calculated for a fixed set of sample points on theslide. These focus values are then used as input to the focus profilecalculation, prior to scanning the slide.

With the dynamic pre-focus procedure, focusing is interleaved withscanning. As in the static approach, either the ribbon-focus or thepoint-focus method is used. Prior to scanning a particular stripe, onlythose sample points that directly affect the focus profile calculationfor that stripe are focused. Sample points that have recently beenfocused for the purpose of scanning a neighboring stripe will not needto be refocused.

Two advantages are gained by the dynamic approach. First, the focusingcan be done in parallel with other scanning tasks, such as disk readsand writes. Additionally, underutilized processor and motion controlresources can be used without adding significantly to the scan time.Moreover, focus drift is greatly reduced, since the time between focusand acquisition is reduced.

In step 250, the system determines if the static method or the dynamicmethod is to be employed. If it is the static method, then the systemproceeds to get the focus values for all stripes (the entire scan area),as shown in step 260. If it is the dynamic method, then the systemproceeds to get the focus values for the next stripe to be scanned, asshown in step 270.

The point-focus method utilizes the set of points resulting from thefocus point placement described in step 210. For each focus point, theslide is positioned so that the optical axis, and hence the center ofthe line scan camera, is centered on that point. The height of theobjective lens is adjusted in small steps through a pre-defined range oftravel. At each step, a frame of imagery data is collected from the linescan camera and a contrast metric is computed by summing the squareddifferences between neighboring pixel values. The height having thelargest contrast value is taken to be the correct focus height. In orderto reduce the number of steps that must be considered, the planecalculated from the approximated focal surface described in step 230 isused to set the mid-point of the range.

In the ribbon-focus method, successive frames of camera data arecollected while the slide is in motion, much like a regular scan. Duringthe scan, the objective lens follows an oscillatory height profile,which is synchronized with slide position. At each point in the scan,both the height of the objective lens and its planar position are known,however, the image data at all points will not be in focus. A contrastvalue is computed for each frame of camera data. The contrast value willpeak at those locations along the path of the objective lens where theobjective is scanning at the optimal focus height. These peaks areidentified and the planar position and height are recorded as a validfocus point.

The exact nature of the oscillatory height profile is not as importantas synchronization with slide position. Any number of profiles can beused, including saw-tooth, triangular, and sinusoidal. The sinusoidalprofile has the advantage of smooth acceleration and deceleration atmotion limits, but is more complex to program. The saw-tooth profile hasthe advantage of simplicity.

If it is determined in step 250 that static pre-focus is being used, anumber of ribbon-focus scans can be done to generate focus point valuesdistributed throughout the scan area. If it is determined in step 250that dynamic pre-focus is being used, it is preferred to use thetrajectory of the next stripe as the path for the ribbon-focus scan. Inthis way, the focus values will lie exactly on that part of the slide tobe scanned next. It is also possible to analyze the contrast value fordifferent segments of the camera pixel array in order to estimatevariations in focus height across its width. This additional informationcan be used to estimate local tilt of the slide and used as input to amechanical apparatus for tilt compensation.

When collecting a stripe of image data, the objective lens follows aheight profile derived from the sampled focus points and calculated intoa focus profile, as shown in step 280. The focus profile can be for asingle stripe, as in the case of dynamic pre-focus or the focus profilecan be for the entire slide (or specimen occupying sub-region thereof).A preferred method for calculating the focus profile is to useinterpolation based upon Delaunay triangulation. Accordingly, a surfacecomprised of a set of triangular facets is generated by connecting eachfocus point with a pair of neighboring focus points. The benefit ofusing Delaunay triangulation is that it generates the set of triangleshaving the smallest mean-squared angular sum. In more simple terms,Delaunay triangulation generates triangular facets that have largerinterior angles than other triangulation methods.

To calculate the focus height at a given point on the slide, thetriangle enclosing the coordinates for that point is located. A planarsurface connecting the height values for the vertices of that triangleis then constructed. The desired focus height is obtained by projectingthe given point onto that planar surface. The surface function is then aset of planar triangular facets, joined at the height values of themeasured focus points.

Once the focus profile is calculated, in step 290 the line scan systemscans the next stripe of image data. If more stripes need to be scanned,as determined in step 300, the system returns to step 270 get the focuspoint values for the next stripe in the case of dynamic pre-focus(determined in step 310) or returns to calculate the focus profile forthe next strip in the case of static pre-focus. Once all of the stripeshave been scanned, the process ends, as illustrated in step 320.

FIG. 10 is a block diagram illustrating an exemplary computer system 550that may be used in connection with the various embodiments describedherein. For example, the computer system 550 may be used in conjunctionwith a ScanScope® scanner machine. However, other computer systemsand/or architectures may be used, as will be clear to those skilled inthe art.

The computer system 550 preferably includes one or more processors, suchas processor 552. Additional processors may be provided, such as anauxiliary processor to manage input/output, an auxiliary processor toperform floating point mathematical operations, a special-purposemicroprocessor having an architecture suitable for fast execution ofsignal processing algorithms (e.g., digital signal processor), a slaveprocessor subordinate to the main processing system (e.g., back-endprocessor), an additional microprocessor or controller for dual ormultiple processor systems, or a coprocessor. Such auxiliary processorsmay be discrete processors or may be integrated with the processor 552.

The processor 552 is preferably connected to a communication bus 554.The communication bus 554 may include a data channel for facilitatinginformation transfer between storage and other peripheral components ofthe computer system 550. The communication bus 554 further may provide aset of signals used for communication with the processor 552, includinga data bus, address bus, and control bus (not shown). The communicationbus 554 may comprise any standard or non-standard bus architecture suchas, for example, bus architectures compliant with industry standardarchitecture (“ISA”), extended industry standard architecture (“EISA”),Micro Channel Architecture (“MCA”), peripheral component interconnect(“PCI”) local bus, or standards promulgated by the Institute ofElectrical and Electronics Engineers (“IEEE”) including IEEE 488general-purpose interface bus (“GPIB”), IEEE 696/S-100, and the like.

Computer system 550 preferably includes a main memory 556 and may alsoinclude a secondary memory 558. The main memory 556 provides storage ofinstructions and data for programs executing on the processor 552. Themain memory 556 is typically semiconductor-based memory such as dynamicrandom access memory (“DRAM”) and/or static random access memory(“SRAM”). Other semiconductor-based memory types include, for example,synchronous dynamic random access memory (“SDRAM”), Rambus dynamicrandom access memory (“RDRAM”), ferroelectric random access memory(“FRAM”), and the like, including read only memory (“ROM”).

The secondary memory 558 may optionally include a hard disk drive 560and/or a removable storage drive 562, for example a floppy disk drive, amagnetic tape drive, a compact disc (“CD”) drive, a digital versatiledisc (“DVD”) drive, etc. The removable storage drive 562 reads fromand/or writes to a removable storage medium 564 in a well-known manner.Removable storage medium 564 may be, for example, a floppy disk,magnetic tape, CD, DVD, etc.

The removable storage medium 564 is preferably a computer readablemedium having stored thereon computer executable code (i.e., software)and/or data. The computer software or data stored on the removablestorage medium 564 is read into the computer system 550 as electricalcommunication signals 578.

In alternative embodiments, secondary memory 558 may include othersimilar means for allowing computer programs or other data orinstructions to be loaded into the computer system 550. Such means mayinclude, for example, an external storage medium 572 and an interface570. Examples of external storage medium 572 may include an externalhard disk drive or an external optical drive, or and externalmagneto-optical drive.

Other examples of secondary memory 558 may include semiconductor-basedmemory such as programmable read-only memory (“PROM”), erasableprogrammable read-only memory (“EPROM”), electrically erasable read-onlymemory (“EEPROM”), or flash memory (block oriented memory similar toEEPROM). Also included are any other removable storage units 572 andinterfaces 570, which allow software and data to be transferred from theremovable storage unit 572 to the computer system 550.

Computer system 550 may also include a communication interface 574. Thecommunication interface 574 allows software and data to be transferredbetween computer system 550 and external devices (e.g. printers),networks, or information sources. For example, computer software orexecutable code may be transferred to computer system 550 from a networkserver via communication interface 574. Examples of communicationinterface 574 include a modem, a network interface card (“NIC”), acommunications port, a PCMCIA slot and card, an infrared interface, andan IEEE 1394 fire-wire, just to name a few.

Communication interface 574 preferably implements industry promulgatedprotocol standards, such as Ethernet IEEE 802 standards, Fiber Channel,digital subscriber line (“DSL”), asynchronous digital subscriber line(“ADSL”), frame relay, asynchronous transfer mode (“ATM”), integrateddigital services network (“ISDN”), personal communications services(“PCS”), transmission control protocol/Internet protocol (“TCP/IP”),serial line Internet protocol/point to point protocol (“SLIP/PPP”), andso on, but may also implement customized or non-standard interfaceprotocols as well.

Software and data transferred via communication interface 574 aregenerally in the form of electrical communication signals 578. Thesesignals 578 are preferably provided to communication interface 574 via acommunication channel 576. Communication channel 576 carries signals 578and can be implemented using a variety of communication means includingwire or cable, fiber optics, conventional phone line, cellular phonelink, radio frequency (“RF”) link, or infrared link, just to name a few.

Computer executable code (i.e., computer programs or software) is storedin the main memory 556 and/or the secondary memory 558. Computerprograms can also be received via communication interface 574 and storedin the main memory 556 and/or the secondary memory 558. Such computerprograms, when executed, enable the computer system 550 to perform thevarious functions of the present invention as previously described.

In this description, the term “computer readable medium” is used torefer to any media used to provide computer executable code (e.g.,software and computer programs) to the computer system 550. Examples ofthese media include main memory 556, secondary memory 558 (includinghard disk drive 560, removable storage medium 564, and external storagemedium 572), and any peripheral device communicatively coupled withcommunication interface 574 (including a network information server orother network device). These computer readable mediums are means forproviding executable code, programming instructions, and software to thecomputer system 550.

In an embodiment that is implemented using software, the software may bestored on a computer readable medium and loaded into computer system 550by way of removable storage drive 562, interface 570, or communicationinterface 574. In such an embodiment, the software is loaded into thecomputer system 550 in the form of electrical communication signals 578.The software, when executed by the processor 552, preferably causes theprocessor 552 to perform the inventive features and functions previouslydescribed herein.

Various embodiments may also be implemented primarily in hardware using,for example, components such as application specific integrated circuits(“ASICs”), or field programmable gate arrays (“FPGAs”). Implementationof a hardware state machine capable of performing the functionsdescribed herein will also be apparent to those skilled in the relevantart. Various embodiments may also be implemented using a combination ofboth hardware and software.

While the particular systems and methods herein shown and described indetail are fully capable of attaining the above described objects ofthis invention, it is to be understood that the description and drawingspresented herein represent a presently preferred embodiment of theinvention and are therefore representative of the subject matter whichis broadly contemplated by the present invention. It is furtherunderstood that the scope of the present invention fully encompassesother embodiments that may become obvious to those skilled in the artand that the scope of the present invention is accordingly limited bynothing other than the appended claims.

The invention claimed is:
 1. A computer implemented method for achievingfocus in a digital pathology system having an objective lens coupled toa line scan camera and a stage for supporting a microscope slide, whereone or more processors of the digital pathology system are programmed toperform steps comprising: identifying a plurality of focus points on amicroscope slide; positioning an objective lens coupled with a line scancamera over a first focus point; scanning an image of the first focuspoint in at least three separate series, wherein each series comprisesat least ten different objective lens heights having a distance betweeneach objective lens height and wherein said distance between eachobjective lens height decreases for each successive series; anddetermining the objective lens height having the greatest contrast inthe scanned image.
 2. The method of claim 1, wherein determining theobjective lens height having the greatest contrast in the scanned imagecomprises selecting the objective lens height having the greatestcontrast in the scanned image from the last series.
 3. The method ofclaim 1, wherein the distance between objective lens heights in the atleast three separate series is measured in microns.
 4. The method ofclaim 1, wherein the distance between objective lens heights in a firstseries of the at least three separate series is between 3 and 7 microns.5. The method of claim 1, wherein the distance between objective lensheights in a second of the at least three separate series is between 0.5and 2.9 microns.
 6. The method of claim 1, wherein the distance betweenobjective lens heights in a third of the at least three separate seriesis between 0.1 and 0.4 microns.
 7. A system for creating a digital imageof a specimen on a microscope slide, comprising: a stage configured tosupport a microscope slide having a specimen; an objective lens; amotion control system configured to move the stage and adjust the heightof the objective lens relative to the stage while the stage is inmotion; a line scan camera coupled with the objective lens, wherein theline scan camera is configured to scan an image of an area of thespecimen while the stage is moving; a processor configured to determinea plurality of focus points in a scan area of the microscope slide andscan images of each focus point in at least three separate series,wherein each series comprises at least ten different objective lensheights having a distance between each objective lens height and whereinsaid distance between each objective lens height decreases for eachsuccessive series and wherein the processor is configured to determinethe objective lens height having the greatest contrast in the scannedimage.
 8. The system of claim 7, wherein the objective lens heighthaving the greatest contrast in the scanned image is from the lastseries.
 9. The system of claim 7, wherein the distance between objectivelens heights in the at least three separate series is measured inmicrons.
 10. The system of claim 7, wherein the distance betweenobjective lens heights in a first series of the at least three separateseries is between 3 and 7 microns.
 11. The system of claim 7, whereinthe distance between objective lens heights in a second of the at leastthree separate series is between 0.5 and 2.9 microns.
 12. The system ofclaim 7, wherein the distance between objective lens heights in a thirdof the at least three separate series is between 0.1 and 0.4 microns.